26 research outputs found
Accurate Optical Flow via Direct Cost Volume Processing
We present an optical flow estimation approach that operates on the full
four-dimensional cost volume. This direct approach shares the structural
benefits of leading stereo matching pipelines, which are known to yield high
accuracy. To this day, such approaches have been considered impractical due to
the size of the cost volume. We show that the full four-dimensional cost volume
can be constructed in a fraction of a second due to its regularity. We then
exploit this regularity further by adapting semi-global matching to the
four-dimensional setting. This yields a pipeline that achieves significantly
higher accuracy than state-of-the-art optical flow methods while being faster
than most. Our approach outperforms all published general-purpose optical flow
methods on both Sintel and KITTI 2015 benchmarks.Comment: Published at the Conference on Computer Vision and Pattern
Recognition (CVPR 2017
Feedback MPC for Torque-Controlled Legged Robots
The computational power of mobile robots is currently insufficient to achieve
torque level whole-body Model Predictive Control (MPC) at the update rates
required for complex dynamic systems such as legged robots. This problem is
commonly circumvented by using a fast tracking controller to compensate for
model errors between updates. In this work, we show that the feedback policy
from a Differential Dynamic Programming (DDP) based MPC algorithm is a viable
alternative to bridge the gap between the low MPC update rate and the actuation
command rate. We propose to augment the DDP approach with a relaxed barrier
function to address inequality constraints arising from the friction cone. A
frequency-dependent cost function is used to reduce the sensitivity to
high-frequency model errors and actuator bandwidth limits. We demonstrate that
our approach can find stable locomotion policies for the torque-controlled
quadruped, ANYmal, both in simulation and on hardware.Comment: Paper accepted to IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS 2019
Frequency-Aware Model Predictive Control
Transferring solutions found by trajectory optimization to robotic hardware
remains a challenging task. When the optimization fully exploits the provided
model to perform dynamic tasks, the presence of unmodeled dynamics renders the
motion infeasible on the real system. Model errors can be a result of model
simplifications, but also naturally arise when deploying the robot in
unstructured and nondeterministic environments. Predominantly, compliant
contacts and actuator dynamics lead to bandwidth limitations. While classical
control methods provide tools to synthesize controllers that are robust to a
class of model errors, such a notion is missing in modern trajectory
optimization, which is solved in the time domain. We propose frequency-shaped
cost functions to achieve robust solutions in the context of optimal control
for legged robots. Through simulation and hardware experiments we show that
motion plans can be made compatible with bandwidth limits set by actuators and
contact dynamics. The smoothness of the model predictive solutions can be
continuously tuned without compromising the feasibility of the problem.
Experiments with the quadrupedal robot ANYmal, which is driven by
highly-compliant series elastic actuators, showed significantly improved
tracking performance of the planned motion, torque, and force trajectories and
enabled the machine to walk robustly on terrain with unmodeled compliance
Deep Drone Racing: From Simulation to Reality with Domain Randomization
Dynamically changing environments, unreliable state estimation, and operation
under severe resource constraints are fundamental challenges that limit the
deployment of small autonomous drones. We address these challenges in the
context of autonomous, vision-based drone racing in dynamic environments. A
racing drone must traverse a track with possibly moving gates at high speed. We
enable this functionality by combining the performance of a state-of-the-art
planning and control system with the perceptual awareness of a convolutional
neural network (CNN). The resulting modular system is both platform- and
domain-independent: it is trained in simulation and deployed on a physical
quadrotor without any fine-tuning. The abundance of simulated data, generated
via domain randomization, makes our system robust to changes of illumination
and gate appearance. To the best of our knowledge, our approach is the first to
demonstrate zero-shot sim-to-real transfer on the task of agile drone flight.
We extensively test the precision and robustness of our system, both in
simulation and on a physical platform, and show significant improvements over
the state of the art.Comment: Accepted as a Regular Paper to the IEEE Transactions on Robotics
Journal. arXiv admin note: substantial text overlap with arXiv:1806.0854
High Speed and High Dynamic Range Video with an Event Camera
Event cameras are novel sensors that report brightness changes in the form of
a stream of asynchronous "events" instead of intensity frames. They offer
significant advantages with respect to conventional cameras: high temporal
resolution, high dynamic range, and no motion blur. While the stream of events
encodes in principle the complete visual signal, the reconstruction of an
intensity image from a stream of events is an ill-posed problem in practice.
Existing reconstruction approaches are based on hand-crafted priors and strong
assumptions about the imaging process as well as the statistics of natural
images. In this work we propose to learn to reconstruct intensity images from
event streams directly from data instead of relying on any hand-crafted priors.
We propose a novel recurrent network to reconstruct videos from a stream of
events, and train it on a large amount of simulated event data. During training
we propose to use a perceptual loss to encourage reconstructions to follow
natural image statistics. We further extend our approach to synthesize color
images from color event streams. Our network surpasses state-of-the-art
reconstruction methods by a large margin in terms of image quality (> 20%),
while comfortably running in real-time. We show that the network is able to
synthesize high framerate videos (> 5,000 frames per second) of high-speed
phenomena (e.g. a bullet hitting an object) and is able to provide high dynamic
range reconstructions in challenging lighting conditions. We also demonstrate
the effectiveness of our reconstructions as an intermediate representation for
event data. We show that off-the-shelf computer vision algorithms can be
applied to our reconstructions for tasks such as object classification and
visual-inertial odometry and that this strategy consistently outperforms
algorithms that were specifically designed for event data.Comment: arXiv admin note: substantial text overlap with arXiv:1904.0829